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The  Problem 


TBE  wanted  to  improve  our  TML+  mortality  algorithm  by  factoring  in  the  chance 
of  death  as  a  function  of  time  based  on  real-world  data. 


The  current  DOW  mortality 
algorithm  is  based  on 
responses  from  a  panel  of 
medical  SMEs  rather  than 
empirical  data 

Due  to  time  limitations  and 
the  amount  of  data,  we 
decided  to  pursue  an 
automated  method  for 
assigning  patient  condition 
codes 


Weibull  Survival  Function  Applied  to 
Stages  of  No  Trtmt,  1stR  and  Next  LOC 
after  1stR  (High  Risk) 


d) 


Time,  t  (hrs) 


Source  Data 


The  Combat  Trauma  Registry  (CTR)  database  maintained  by  the  Naval  Health 
Research  Center  records  medical  data  from  patients  treated  at  Navy  and  Marine 
Corps  medical  facilities 

Each  CTR  record  documents  a  single  visit  (encounter)  by  a  single  patient 

Each  record  contains  over  50  individual  fields  of  data,  though  often  the  data  is 
incomplete 

The  most  relevant  data  (such  as  injury  descriptions)  are  primarily  free  text 
containing  numerous  abbreviations,  medical  terms,  and  misspellings 

Example: 

“FOREHEAD  LAC  X  2  NASAL  FX  DEFORMITY  LT  SUPERIOR  ORBITAL 
RIDGE  FX  LT  CORNEAL  ABRASION  WITH  FOREIGN  BODY--” 


Analysis  Method  Benefits 


Subject  matter  expert 
Strengths 

•  True  medical  knowledge 

•  Recognition  of  misspelled  words, 
abbreviations,  and  acronyms 

Weaknesses 

•  Subjective  -  different  SMEs  may 
give  different  results 

•  Easy  to  miss  important  data 

•  Slow  (several  minutes  per 
encounter) 

•  Expensive 


Machine  data  mining 

Strengths 

•  Inexpensive 

•  Consistent  results 

•  Very  fast 

Weaknesses 

•  No  real  medical  knowledge,  only 
rules 

•  Accuracy  is  reduced  by  unusual 
words,  abbreviations,  acronyms 

•  Extremely  difficult  to  develop 
algorithm 


step  1 :  Clean  and  Associate  the  Data 


For  privacy  protection,  NHRC  replaced  SSNs  with  another  unique  identifier 

Each  CTR  record  represents  a  single  CTR  form.  One  is  filled  out  in  the  field  after 
each  patient  encounter. 


There  may  be  multiple  encounters  (and  forms)  for  a  single  person  at  different 


locations 

A  patient  record  was 
created  to  keep  track 
of  the  relationship 
between  multiple 
encounters  and  a 
single  patient 
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step  2:  Catalog  Words 


The  algorithm  begins  by  reading  all  text  fields  and  cataloging  every  word  and  it’s 
associated  encounter  record.  For  example,  imagine  this  is  in  the  injury  description 


FOREHEAD  LAC  X  2  NASAL  FX  DEFORMITY  LT 
SUPERIOR  ORBITAL  RIDGE  FX  LT  CORNEAL 
ABRASION  WITH  FOREIGN  BODY- 


The  algorithm  generates  the  following  initial  word  list: 


FOREHEAD 

LAC 

X 

2 

NASAL 

FX 

DEFORMITY 

LT 


•SUPERIOR 

•ORBITAL 

•RIDGE 

•CORNEAL 

•ABRASION 

•WITH 

•FOREIGN 

•BODY 


step  3:  Simplify/Consolidate  Word  List 


The  algorithm  continues  by  eliminating  common  “noise”  words  and  translating 
abbreviations  it  recognizes  into  full  words.  It  also  simplifies  some  words  that  are 
too  specific  for  the  PC  code  description.  For  instance,  in  this  case,  it  changes 
“forehead”  to  “head”  since  there  are  no  forehead-specific  injuries  in  the  PC  list: 


•FOREHEADS  HEAD 
•LAC  ^LACERATION 
•X  (single-letter  word) 

•2  ::  J:  (single-letter word) 
•NASALS  NOSE 
•FX^  FRACTURE 
•DEFORMITY 
•LT^  LEFT 


•SUPERIOR 
•ORBITALS  HEAD 
•RIDGE 

•CORNEAL^  EYE 
•ABRASION 

•WITH  (common  word) 

•FOREIGN 

•BODY 


Note  one  difficulty  already: 

•“X  2”  was  eliminated,  even  though  it  was  specifying  the  number  of  lacerations, 
which  may  be  important  for  diagnosis 


FOREHEAD  LAC  X  2  NASAL  FX  DEFORMITY  LT 
SUPERIOR  ORBITAL  RIDGE  FX  LT  CORNEAL 
ABRASION  WITH  FOREIGN  BODY- 
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step  4:  Associate  Related  Words 


A  specific  list  of  adjectives  are  used  by  the  algorithm  to  keep  some  important 
phrases  together.  In  this  case,  “LEFT”  is  one  of  those  words  that  must  be  kept  with 
the  word  next  to  it,  or  it  loses  all  relevance  to  the  word  matching  algorithm. 

Phrases  are  treated  the  same  as  single  words  from  this  point  on. 


So,  the  word  list  becomes  the  following: 

•HEAD 

•LACERATION 

•NOSE 

•FRACTURE 

•DEFORMITY 

•LEFT  SUPERIOR 

•HEAD 


•RIDGE 
•LEFT  EYE 
•ABRASION 
•FOREIGN 
•BODY 


Another  difficulty: 

•“LEFT”  came  before  “SUPERIOR”  so  they  were  grouped  together  as  a  phrase, 
however  the  phrase  “LEFT  SUPERIOR”  really  describes  the  word  “HEAD”  after  it. 
The  algorithm  currently  does  not  support  phrases  longer  than  two  words. 


FOREHEAD  LAC  X  2  NASAL  FX  DEFORMITY  LT 
SUPERIOR  ORBITAL  RIDGE  FX  LT  CORNEAL 
ABRASION  WITH  FOREIGN  BODY- 
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step  5:  Assign  Weights 


Each  word  (or  phrase)  is  assigned  a  numeric  value  based  on  the  location  in  which 
it  was  found.  For  instance,  words  from  the  injury  description  have  a  higher  weight 
than  words  from  the  SOAP  notes,  which  tend  to  tell  more  about  the  treatment  than 
the  injury.  Multiple  instances  of  the  same  word  are  counted  separately  then  added 
together. 

Also  some  words  are  given  higher  weights  because  they  are  highly  relevant  when 
determining  the  patient  condition  (these  are  usually  related  to  the  anatomical 
location). 

Examples: 


HEAD  (0  +  100  (found  in  inj  desc)  +  400 

100  (highly  relevant  word))  X 
2  (two  instances) 

RIDGE  (0  +  100  (found  in  inj  desc))  100 


FOREHEAD  LAC  X  2  NASAL  FX  DEFORMITY  LT 
SUPERIOR  ORBITAL  RIDGE  FX  LT  CORNEAL 
ABRASION  WITH  FOREIGN  BODY- 
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step  6:  Assign  Category 


•  PC  descriptions  are  parsed  in  a  similar  manner 

•  “Instant  category  match”  words  are  considered  (such  as 
HEAD).  These  limit  the  PC  choice  to  those  in  the  head 
category 

•  Anatomical  location  is  considered  to  further  limit  PC 
category,  if  possible 

•  Keywords  (such  as  KIA,  CPR,  Intubated,  etc.)  used  to 
assign  definitely/probably/maybe  life  threatening  to  the 
patient. 

•  Category  is  further  limited  to  only  PCs  that  are  LT  if  the 
patient  is  “definitely”  LT. 


Anatomical  record  fields  checked  “yes” 

Resulting  PC  Category 

*  Neck,  Head,  Face,  Eye,  or  Ear 

Head 

*  Genitalia,  Abdomen,  or  Pelvis 

Abdomen  &  Pelvis 

*  Back 

Spine 

*  Thorax/Back  or  Chest 

Thorax 

Lower  Extremity 

Lower  Limbs 

Upper  Extremity 

Upper  Limbs 

Two  or  more  of  those  criteria  marked  with  * 
above 

Multiple  Injury  Wounds 

HEAD 

400 

LACERATION 

100 

NOSE 

200 

FRACTURE 

200 

DEFORMITY 

100 

LEFT  SUPERIOR 

100 

RIDGE 

100 

LEFT  EYE 

200 

ABRASION 

100 

FOREIGN 

100 

BODY 

100 

Word/weight  list  for  our  sample 


FOREHEAD  LAC  X  2  NASAL  FX  DEFORMITY  LT 
SUPERIOR  ORBITAL  RIDGE  FX  LT  CORNEAL 
ABRASION  WITH  FOREIGN  BODY- 
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step  7:  Generate  PC  Match  Ranking  List 


Now,  each  encounter’s  word  list 
is  compared  to  each  PC 
description  word  list.  The 
weights  for  all  matches  are 
added  together  and  divided  by 
the  number  of  relevant  words  in 
the  PC  description.  This  keeps 
long  PC  descriptions  from 
matching  more  often  than  short 
ones  simply  because  they  have 
more  words. 

The  top  three  match  indexes  are 
reported  to  the  analyst. 


PC  Description  Word  List  (PC  10)  Example  Encounter  Word  List 


HEAD 

300 

400 

CONTUSION 

100 

LACERATION 

100 

OPEN 

200 

NOSE 

200 

FRACTURE 

200 

- ^-FRACTURE 

200 

MODERATE 

100 

DEFORMITY 

100 

NO  HEAD 

100 

LEFT  SUPERIOR 

100 

FRAGMENT 

100 

RIDGE 

100 

DEPRESSED 

200 

LEFT  EYE 

200 

ABRASION 

100 

300+400+200+200 

=  1100 

FOREIGN 

100 

1100/8  =  137.5 

BODY 

100 

Our  encounter  matches  PC 
10  with  a  confidence  index 
of  137.5. 


FOREHEAD  LAC  X  2  NASAL  FX  DEFORMITY  LT 
SUPERIOR  ORBITAL  RIDGE  FX  LT  CORNEAL 
ABRASION  WITH  FOREIGN  BODY- 
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How  did  it  do? 


The  algorithm  was  designed  against  a  list  of  53  CTR  patient  records  that  had 
already  had  a  PC  assigned  by  NHRC. 


53  NHRC  records  (already  had  PC  assigned 
by  NHRC) 

91% 

45% 

37  records  algorithm  determined  to  be 
abdomen  &  pelvis  category  and  “defintely”  LT 

89% 

70% 

141  MIW  records 

~69%* 

41% 

*  Counted  all  definite  matches,  “right  category/wrong  PC”  matches,  and  “right  category,  NLT”  matches 


During  development,  we  determined  there  was  insufficient  time  to  get  exact  PCs 
and  started  trying  just  for  category  matches. 

MIW  was  the  most  difficult  category  for  the  estimator  to  compare,  since  the 
anatomical  location  is  less  relevant  in  choosing  the  proper  PC. 

Estimator  picked  the  correct  PC  as  one  of  it’s  top  three  almost  50%  of  the  time. 
Not  adequate  for  our  analysis,  but  clearly  has  some  value  as  a  first  step  in  an 
automated  methodology. 


Conclusion  -  Potential  Improvements 


•  Decrease  reliance  on  individual  fields  (like  anatomical  location)  in  favor  of  free- 
text  fields  like  injury  description.  They  are  harder  to  parse  but  relying  too  much  on 
individual  fields  may  be  misleading  if  data  is  bad. 

•  Add  adjective  recognition,  keep  adjectives  with  the  words  they  describe.  Support 
complex  descriptive  sentence  structures,  like  “superficial  3-inch  thigh  laceration.” 

•  Severity  ranking  -  attempt  to  determine  severity  of  injuries  and  match  with  only 
the  most  severe 

•  Add  “medical  visualization”  allowing  algorithm  to  understand  that  some  types  of 
injuries,  due  to  their  location,  may  impact  nearby  organs. 

•  Add  a  rudimentary  expert  system  with  some  medical  knowledge  rules.  For 
instance,  if  blood  was  used,  the  injury  was  probably  life  threatening. 

•  Expand  to  more  complex  coding  systems,  like  ICD-9  and  ICD-10.  This  could  be 
used  for  medical  coding  or  for  building  a  PC  to  ICD  mapping. 
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Conclusion  -  Contact  information 


Email:  ioseph.parker@tbe.com 

Fora  more  detailed  technical  description  of  the  algorithm,  word  lists,  etc. 


Visit  www.tmlsim.com 

For  more  information  on  TML+  and  the  DOW  algorithm. 
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Backups 
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Extracting  Mortality  Related  Data  from  the  CTR  File 


Receive  CTR  data  from  NHRC,  extract  patients 
and  encounters,  fix  date  formatting  (3790 
encounters) 


Keep  only  records  with  adequate  timing  data 
(2961  encounters) 


Perform  automated  LT  analysis,  keep  only  LT 
records  (554  encounters) 


Adequate  Timini 

3  Data? 

Kept  for 
DOW 
study 

Injury 

Arrival 

Disposition 

No 

No 

No 

No 

No 

Yes 

No 

No 

No 

No 

Yes 

No 

No 

Yes 

Yes 

Yes 

y  Yes 

No 

No 

No 

l/Yes 

Yes 

No 

Yes 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

Yes 

Injury  time,  arrival  time,  and  disposition  time 
are  available  in  the  CTR  data,  but  not  every 
encounter  has  all  data.  Records  were  kept 
based  on  the  criteria  in  the  table  above. 


SMEs  were 
provided  an 
application  to 
evaluate  LT 
status  of  each 
patient 


Give  patient/encounter  list  to  SMEs,  asking  for 
NLT,  High,  Med,  Low,  Not  enough  info  rating  for 
each  patient  (436  patients,  554  encounters) 


4  CTKOOWAmI/mTmI 


±z 


ooyy/MyMUwnt 

Ptiwib  H  4  2  0(42^  P  h 


Reconcile  and  merge  SME  results,  export  Excel 
worksheet  for  analysis  (Resulting  patients:  158  H, 
42  M,  44  L,  192  Other) 


(mounim  .  • 
FiMCTRAkoiP 
Fern  S  FRK19  Wi 


TMtO/nvri  2^-3096  lOOMM 


>  MO  NS  BOLUS  0400  RT  MM  MOPAQ  B  M 

71  ua«.  \n/%  fwe  fttf  vnTH  pfiessuflc  , 
ORTSSMCrr  riMORAD 

SF3ErX>MALEA<OX3,LeFTNMODOMnUNT  W1TM  B 
NKTI  FRMj  WOUWS  BEtfi  HIT  BYGKNACf  T 
CM  entry  WOUND  IM4CR  ASPECT  Of  NCDM.THCH  I  J 
PALMOf  LT  NANOIRT  tM.COPAmmRT  TMOHt 
HAND  PAW  .7/»0l  AfTB»  MEI>$ 

0)  PAIRABU  PULSED  AUEXTR  NO  DGCMEASS  BLOOD  . 


Owt  SUPtflfCMtWOUWOVERLT  CtAVUtCTABLA  LT  PALM  VHTMtNTRANCt  WOUND  TWNMPROMWtNC 

Manx  loniM  RT  IRD  POH  THOH  WITH  ENTRANCE  WOUND  NON  PU 

wdtwck  RT  EAR  <  I  CM  SUPERfOAL  WOUND  MO ONOOMSBlEEDWa  NO  palpable  PRAG 


Algorithm  is  described  in  detail,  including  all  LT  qualifiers,  in  the 
4/11/2007  document  “DOW  PC  Estimator  Algorithm  Summary  4- 
11-07.doc” 


Automated  LT  algorithm 
determined  likely  LT 
encounters  based  on 
criteria  from  12/12/06 
NHRC  and  TBE  technical 
interchange  meeting. 

Criteria  consisted  of  words 
and  phrases  found 
anywhere  in  the  encounter 
text  (such  as  “Pos  FAST” 
or  “Deceased”)  and 
qualifiers  in  particular  fields 
(such  as  Hemorrhage  =  11, 
III,  or  IV).  See  full 
algorithm  description 
document  for  more  details. 
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Patient  Condition  Code  Sampie 


Description 

CEREBRAL  CONTUSION  WITH  OPEN  SKULL  FRACTURE  MODERATE  -  WITHOUT  INTRACRANIAL 
FRAGMENTS  AND/OR  DEPRESSED  SKULL  FRACTURE 

-  scalped 

©^i^WfriouT  cerebral  injury  or  skull  fracture  moderate  -  scalp 

LACERATION 

FRACTURE  FACIAL  BONES  CLOSED  EXCLUSIVE  OF  MANDIBLE  SEVERE  -  MULTIPLE  FRACTURES 
FRACTURE  FACIAL  BONES  CLOSED  EXCLUSIVE  OF  MANDIBLE  MODERATE  -  SINGLE  FRACTURE 

WOUND  FACE  JAWS  AND  NECK  OPEN  LACERATED  WITH  ASSOCIATED  FRACTURES  EXCLUDING 
SPINAL  FRACTURES  SEVERE  -  WITH  AIRWAY  OBSTRUCTION 

WOUND  FACE  JAWS  AND  NECK  OPEN  LACERATED  WITH  ASSOCIATED  FRACTURES  EXCLUDING 
SPINAL  FRACTURES  MODERATE  -  WITHOUT  AIRWAY  OBSTRUCTION;  EYELID  AND  EYEBALL 
LACERATION  WITH  RETAINED  INTRAOCULAR  FOREIGN  BODY 

WOUND  FACE  AND  NECK  OPEN  LACERATED  CONTUSED  WITHOUT  FRACTURES  SEVERE  -  WITH 
AIRWAY  OBSTRUCTIONS  AND/OR  MAJOR  VESSEL  INVOLVEMENT 

WOUND  FACE  AND  NECK  OPEN  LACERATED  CONTUSED  WITHOUT  FRACTURES  MODERATE  - 
WITHOUT  AIRWAY  OBSTRUCTION  OR  MAJOR  VESSEL  INVOLVEMENT 

EYE  WOUND  SEVERE  -  LOSS  OF  INTRAOCULAR  FLUID  WITH/WITHOUT  RETINAL  DETACHMENT, 

DETACHMENT  OR  RETINAL  INJURY  NO 

FOREIGN  BODY  RETAINED  WITHOUT  LOSS  OF  VITREOUS  FLUID  PATIENT  HAS  HYPHEMA  EYE 
SALVAGEABLE 

HEARING  IMPAIRMENT  SEVERE 
HEARING  IMPAIRMENT  MODERATE 

FRACTURE  SPINE  CLOSED  WITHOUT  CORD  DAMAGE  UNSTABLE  LESION 

FRACTURi  SPINi  CLOSiD  WITHQjkDT^OQBtMR^A^Sl^^  RESPIRATORY 

INVOLVEMENT 

FRACTURE  SPINE  CLOSED  WITH  CORD  DAMAGE  BELOW  CERVICAL  SPINE  (PROGRESSIVE) 
FRACTURE  SPINE  OPEN  WITH  CORD  DAMAGE  CERVICAL  SPINE  WITH  RESPIRATORY  DISTRESS 
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Patient  Condition  Categories 


Abdomen  &  Pelvis 
Battle  Fatigue 
Burns 

Cardiovascular 

Directed  Energy  Weapon  Eye 

Lesion 

Environmental 

Eye/Ear  Disease 

Female  Specific 

Gastrointestinal 

General 

Genitourinary 

Head 

Infectious/Parasitic 
Lower  Limbs 
Dental 


Multiple  Injury  Wounds 
Miscellaneous 
Neuropsychiatric 
Not  Assigned 

Preventive  Medicine 

Respiratory 

Sexually  Transmitted  Disease 
Spine 

Sprains  &  Strains 

Superficial/Soft  Tissue 

Surgical 

Thorax 

Upper  Limbs 

Dermatological 

Nuclear,  Biological,  Chemical 
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